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Abstract Body 


Background / Context: The changing nature of the U.S. economy has fostered concerns that 
too few students are successfully completing postsecondary education (Achieve, 2004) . Three 
quarters of those who enter high school graduate within four years, with approximately 70 
percent of those graduates enrolling immediately in some form of postsecondary education (Ross 
et ah, 2012) . Of those who do attend, insufficient numbers complete a degree with only a little 
less than half (49 %) of beginning postsecondary students attaining some sort of a postsecondary 
credential within six years of enrolling (Ross, et ah, 2012) . As a result, there have been 
numerous initiatives to increase the number of students who graduate from high school prepared 
to enroll and progress in postsecondary education. One approach is the Early College High 
School (ECHS or early college) model, small schools that blur the line between high school and 
college. The primary goal of the early college model is to increase the number of students who 
graduate from high school and who continue on to and succeed in college. 

Purpose / Objective / Research Question / Focus of Study: This paper presents results from a 
longitudinal experimental study that is examining the impact of early colleges on students’ 
outcomes in high school and in postsecondary (PS) education, specifically on postsecondary 
enrollment and college credit accrual during and after high school. We also provide an interim 
assessment of the cost and benefit of the model. 

Setting: The schools in this study are located in districts throughout the state of North Carolina. 
They include schools located in rural and urban settings from all regions of the state. 

Population / Participants / Subjects: The sample for this paper includes 716 students who 
applied to six different early colleges^ and enrolled in 9* grade in the 2005-2006, 2006-2007, 
and 2007-2008 school years. These six schools enrolled a total of eight cohorts of students (two 
schools enrolled two different cohorts). Table 1 shows descriptive statistics for this sample. 
Intervention / Program / Practice: The ECHS model is an innovative approach to preparing 
students for college. It represents a comprehensive school reform model that is focused explicitly 
and purposefully on preparing all of its students for college (Edmunds, 2012) . Early colleges 
provide students with concurrent high school and college experiences, in some sense eliminating 
the transition between these two stages of education. Primarily located on the campuses of two- 
or four- year colleges and universities, early colleges are expected to provide an academically 
rigorous course of study with the goal of ensuring that all students graduate with a high school 
diploma and two years of university transfer credit or an associate’s degree. Some schools are 
structured as 4-year schools and some schools are structured to have students complete the 
curriculum in 5 years. Each early college is also expected to implement and exhibit a specific set 
of principles, known as Design Principles, developed by North Carolina New Schools (NCNS — 
the public-private partnership that manages these schools) that represent characteristics of high 
quality high schools. These Design Principles, as articulated by NCNS, are as follows: (1) 
Ensuring that students are ready for college; (2) Instilling powerful teaching and learning in 
schools; (3) Providing high student/staff personalization; (4) Redefining professionalism; (5) 
Eeadership and (6) Implementing a purposeful design (North Carolina New Schools Project, 
2011) . Eigure 1 provides a conceptual model of the these components. 

Research Design: The study uses a longitudinal experimental design in which participating early 


* A total of 19 schools enrolled in the study over time. These six represent the first schools enrolled in the study and 
the only schools whose students have progressed through the beginning of six years after 9* grade with the 
necessary data available for the analyses presented in this paper. 
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colleges had to have more applicants than available slots and had to agree to use a lottery to 
randomly assign students. In each of the school’s lotteries, each student applying to the early 
college was assigned a random number and the list of students was ordered from lowest to 
highest, with the lowest numbers being selected into the early college until all available slots 
were filled. Applicants who were offered enrollment into an ECHS make up the treatment group 
while the rest of the applications who enrolled somewhere else make up the control group. 

Data Collection and Analysis: 

Data Sources. The data used in this paper come from administrative data collected from four 
primary sources: the North Carolina Department of Public Instruction (NCPDI), the National 
Student Clearinghouse (Clearinghouse), the North Carolina Community College System 
(Community College), and the Integrated Postsecondary Education Data System (IPEDS). The 
NCDPI collects data from all schools in North Carolina, including demographic data, test scores, 
behavioral data, transcript data, dropout data, graduation data, and school-level expenditures. 

The Clearinghouse collects data representing approximately 94 percent of students enrolled in 
postsecondary institutions in the United States. The primary data available include enrollment 
by semester, institution in which a student is enrolled, and type and date of degrees received. The 
Community College data includes information about each course taken, how many credit hours 
were attempted, how many credit hours were earned, and whether the course was developmental 
or remedial. Einally, we linked the IPEDS finance data to the NSC data to determine the costs of 
the postsecondary institutions the students in our study were attending. 

Measures. This paper examines the ECHS impact on two primary outcomes on which the interim 
assessment of the cost-effectiveness of the model is based on. The first outcome is concerned 
with enrollment in postsecondary education and captured by two distinct variables: (1) a cross- 
sectional dichotomous measure that pertains to enrollment in a two-year PS institution in a given 
grade of high school and is created using the Community College data and (2) a cross-sectional 
categorical measure created using NSC data that captures whether a student was (i) enrolled in a 
two-year postsecondary institution, (ii) enrolled in a four- year postsecondary institution, or (iii) 
was not enrolled in any postsecondary institutions in each post-high school year. 

The second outcome - cumulative college credits earned during and after high school - is 
also captured by two variables: The first variable records the cumulative number of credits a 
student has earned from a two-year institution, expressed in full-time equivalents (PTEs) or 
number of years of college credits earned during high school. The second variable is holds 
credits earned from a four- year institution, expressed in PTEs over the same time frame. This 
variable is created using the NSC data. 

Comparing costs incurred by students over the course of their postsecondary education may 
not be a fair comparison since early college students do not pay for college courses they have 
taken in high school while traditional students do. Therefore we have opted to conducting the 
cost-benefit analysis from the society’s perspective by examining the effect of ECHS model on 
overall educational spending (not educational spending by any one individual) and societal 
benefits associated with educational attainment. Another important feature of our cost-benefit 
analysis is that we calculate a cost and benefit figure for each student a function of their 
education history and baseline characteristics and use these student-level figures for treatment 
and control students to estimate the effect of ECHS on the average societal cost and benefits of 
educational attainment and use the model’s effect on the net benefits (benefit minus cost) to 
assess its cost-effectiveness. This analysis is described in more detail below. 
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In order to calculate educational costs as accurately as possible for each student, we 
determine high schools and two- and four- year colleges he/she attended, the cost of these 
institutions, and the amount of time he/she spent in each institution. The total cost for him/her is 
the sum of his/her high school cost, the cost of community college courses he/she took while in 
high school, and the cost of the two-or four- year institutions he/she attended post high school. 
Figure 2a shows the details of this calculation. 

Societal benefit figures for each student are based on Carroll and Erkut (2009)’s societal 
benefit estimates provided separately for five educational attainment categories: (1) high school 
dropout (no college); (2) high school dropout (some college); (3) high school graduate (no 
college); (4) high school graduate (some college); and (5) 4-year college graduate. The 
placement of students into the first three categories uses the actual data we have on whether 
they’ve graduated from high school, whether they’ve taken any college credits, and whether 
they’ve graduated from college. Unfortunately we do not have sufficient data to place students 
into the last two categories deterministically so this is based on a prediction analysis. 

Specifically, students who are marked as having graduated from high school (according to the 
NCDPI data), have taken some college credits while in high school and/or after high school 
(according to the Clearinghouse and Community College data), but are not predicted to graduate 
from college (which is explained below) are placed in the fourth group. 

The fifth group includes students who are marked as having graduated from high school 
(according to the NCDPI data), have taken some four- year college credits after high school 
(according to the Clearinghouse), and either have graduated from a four- year institution 
(according to the Clearinghouse), or are predicted to graduate from a four- year institution. All 
students who were enrolled in a four- year institution after graduating from high school were 
eligible to graduate from college. According to the National Center for Education Statistics, 58% 
of students complete a four- year degree within six years after enrolling in a four- year institution. 
Slightly fewer than 58 percent of control students who enrolled in a four- year institution after 
college have at least two FTEs at the end of their fourth year after college, so we determined two 
ETEs to be the threshold amount of ETEs a student would need by the end of four years post 
high school in order to graduate from college within six years. We applied this threshold to both 
treatment and control students, so students are predicted to graduate from a four- year institution 
if they are enrolled in a four- year university after high school and have at least two cumulative 
ETEs four years post high school. Figure 2b shows the details of the benefits calculation. 
Statistical Analyses. W e conduct two sets of statistical analyses. The first set are tailored to 
predict students’ cumulative college credits accrued by four years post high school while the 
second set estimate the impact of the early college model on postsecondary outcomes and costs 
and benefits associated with educational attainment. 

We use three separate regression models in the prediction process. The first model uses all 
students with available cumulative FTEs at one and two years post high school (FTEs at PHS 1 
and FTEs at PHS2, respectively) and specifies FTEs at PHS2 as a linear function of FTEs at 
PSHl and student- level baseline covariates including demographics and academic achievement 
in the 8* grade. The second model uses all students with actual FTEs at PHS2 and PHS3 and 
specifies FTEs at PHS3 as a function of FTEs at PHS2 and student covariates. Finally, the third 
model specifies FTEs at PSH4 as a function of FTEs at PHS3 and covariates and is estimated 
using all students with actual accumulated credit measures at PHS3 and PHS4. These models are 
then used to predict all students’ accrued credit at PHS4 (which is then used to calculate the cost 
and benefits as described above). Figure 3 shows data availability and prediction patterns. 
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The impacts of early colleges on the postsecondary outcomes and cost and benefits are 
estimated within an Intent-To-Treat (ITT) framework, in which a student’s initial experimental 
status as a treatment or control student serves as the treatment contrast. The impacts are 
estimated using multivariate linear regression models that include site indicators (or site fixed 
effects), interaction of the treatment indicator with the site indicators (which site level impact 
estimates that are then pooled to yield the overall impact estimate), and baseline student 
characteristics including demographic characteristics such as gender, race/ethnicity, age, 
free/reduced price lunch status, and 8* grade academic performance. 

Findings / Results: Figures 4 through 7 shows the impacts of the model on the four outcome 
variables described above. Specifically, Figure 4 shows that the early colleges have a large and 
statistically significant impact on enrollment in two-year PS institutions while still in high 
school. Figure 5 suggests in the first three years post high school, the four-year enrollment rate in 
the treatment group was larger than the control group (e.g., 39% vs. 27% at PHS2 which was the 
only statistically significant difference) while the two-year enrollment rate in the control group 
was larger than the treatment group (e.g., 41% vs. 19% at PHSl). Figure 6 shows that treatment 
students, on average, accumulated substantially (and statistically significantly) larger number of 
two-year college credits during high school. Figure 7 suggests that credits obtained from two- 
year programs and total college credits were significantly higher for treatment students than 
control students in each year after high school. 

Figure 8 shows that relatively fewer students with similar proportions across the treatment 
and groups are allocated to the first three educational attainment categories. While half of the 
control students were allocated to the fourth group, a noticeably lower proportion of treatment 
students (33 percent) were placed in this group. Finally, there was a considerable difference 
between the proportion of treatment and control students who are placed into the fifth group 
(four- year college graduates): 61 percent of treatment vs. 40 percent of control. 

Figure 9 provides the graphical comparison of societal costs and benefits of educational 
attainment of treatment and control students and also demonstrates the net benefits of the early 
college model. The first two bars in this figure suggest that both the societal costs and benefits 
associated with educational attainment were statistically significantly larger for treatment 
students than control students (treatment cost=$66K, control cost=$55K; treatment 
benefit=$265K, control benefit=$240K). The net benefit for the treatment group ($198K) was 
much larger than the net benefit for the control group ($185K), leading to an average treatment- 
control net benefit difference of $13K, which can be attributed to the early college model given 
the underlying experimental design. Remaining bars in the graph shows this information for 
various subgroups. 

Conclusions: Early results from this study show that the Early College High School model is 
increasing students’ enrollment in postsecondary education, primarily by the required exposure 
in high school. The results show that, as is inherent in its design, the program is successful in 
providing early access to college. This paper also reports results from a detailed cost and benefit 
analysis of the early college model. This analysis demonstrates that the model has a substantial 
impact on societal benefits associated higher educational attainment of early college students 
which offsets the higher costs of the model, yielding a positive impact on the expected net 
benefits to the society. It is important to note that the cost-benefit analysis relies on predicted 
college degree accrued and educational attainment for most students in the sample, which may 
decreased the accuracy of the reported estimates. We will update these results with one more 
year of data by the time of the Spring conference. 
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Appendix B. Tables and Figures 

Figure 1: Early College Theory of Change Relative to College Readiness 

Early College Design Principles Intermediate Outcomes Long Term Outcomes 


College Ready 

Default college preparatory course of study 

Dual and college-credit courses 

Assistance in selecting and applying to college 
courses. 


Powerful Teaching and Learning 

Rigorous and relevant instructional practices 


Personalization 

Academic and affective supports 
Positive staff-student relationships 


Completion of college 
preparatory course of study 


Increased high school 
graduation rates 



Completion of college credits 
while in high school 


Increased enrollment in 
college 



Increased application to 
college 


Increased graduation from 
college 


Purposeful Design 

Location on a college campus 
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Figure 2a: Calculation of Costs 


Cost Component 

Source 

Notes on Calculation 

High School 

NCDPI 

Depends on HSs attended and time spent 

Two-year courses taken in HS 

NCCCS and 
IPEDS 

Sum (2-year courses attempted in HS x Cost of CC ) 

Two-year courses taken post HS 

NSC and 
IPEDS 

Sum (estimated cumulative 2-year credits PHS* x Cost of CC ) 

Four- year courses taken post HS 

NSC and IPEDS 

Sum (estimated cumulative 4-year credits PHS* x Cost of 4- year 
college) 

*For some students, PHS credit accumulation is predicted 


Figure 2b: Calculation of Benefits 


Educational Attainment 

Determination of Status 

High school dropout, no college 

Marked as dropout and not found in NCCCS or NSC 

High school dropout, some college 

Marked as dropout and found in NCCCS and/or NSC. Not enough credits for a 4- 
year degree 

High school graduate, no college 

Marked as HS graduate but not found in NCCCS or NSC 

High school graduate, some college 

Marked as HS graduate and found in NCCCS and/or NSC. Not enrolled in a 4-year 
college and/or predicted college credits at PHS4 do not exceed the threshold 

Bachelor's Degree (or more) 

Marked as HS graduate and enrolled in a 4-year college. Actual/predicted college 
credits at PHS4 do exceed the threshold. 
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Figure 3: Availability of Data 


Year 

Cohort 1 

Cohort 2 

Cohort 3 

4 year 

4 year 

5 year 

4 year 

5 year 

2006 

HSl 





2007 

HS2 

HSl 

HSl 



2008 

HS3 

HS2 

HS2 

HSl 

HSl 

2009 

HS4 

HS3 

HS3 

HS2 

HS2 

2010 

PHS 1 

HS4 

HS4 

HS3 

HS3 

2011 

PHS 2 

PHS 1 

HS5 

HS4 

HS4 

2012 

PHS 3 

PHS 2 

PHS 1 

PHS 1 

HS5 

2013 

PHS 4 

PHS 3 

PHS 2 

PHS 2 

PHS 1 

2014 


PHS 4 

PHS 3 

PHS 3 

PHS 2 

2015 



PHS 4 

PHS 4 

PHS 3 

2016 





PHS 4 

ECHS N 

26 

23 

63 

29 

88 

Control N 

36 

38 

45 

123 

129 


Known 

Predicted 

HS = High School 

PHS = Post High School 
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Percent of Students Enrolled 


Figure 4: Percentage of Students Enrolled in Two-Year College Courses in High School (Cross-Sectional) 


100% n 



9 10 11 12 13 

Grade 


■ Treatment ■ Control 

Control mean is the unadjusted group average and treatment mean is the control mean plus the impact estimate. Statistically 
significant treatment-control differences at the p<0.05 level are shown by *. 
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Percent of Students Enrolled 


Figure 5: Percentage of Students Enrolled in Two and Four-Year College Courses After High School (Cross-Sectional) 



Control mean is the unadjusted group average and treatment mean is the control mean plus the impact estimate. Statistically 
significant treatment-control differences at the p<0.05 level are shown by *. 
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Figure 6: Cumulative FTE Years of Two-Year College Credits Earned in High School 


1.8 n 



9 10 11 12 13 


Grade 

■ Treatment ■ Control 

Control mean is the unadjusted group average and treatment mean is the control mean plus the impact estimate. Statistically 
significant treatment-control differences at the p<0.05 level are shown by *. 
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Cumulative FTEs 


Figure 7: Cumulative FTE Years of College Credits Earned After High School by Two- and Four- Year Institutions 


4.5 


3.5 - 


3 - 


2 . 1 * 


2.5 - 


2 - 


1.5 - 


1 - 


0.5 - 


1 . 8 * 


0.5 


0.3 0.3 


1.7* 



1.7* 


1.1 




Years After HS 


s Control: 4-Year ■ Control: 2-Year ■ Treatment: 4-Year Treatment: 2-Year 


Control mean is the unadjusted group average and treatment mean is the control mean plus the impact estimate. Statistically 
significant treatment-control differences at the p<0.05 level are shown by *. 
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Figure 8: Predicted Educational Attainment of Treatment and Control Students 


70 % -| 
60 % - 
50 % - 
40 % - 
30 % - 
20 % - 



HS Dropout (no college) HS Dropout (some 

college) 


50 % 



HS Graduate (no college) HS Graduate (some 

college) 


■ Treatment ■ Control 


SREE Spring 2015 Conference Abstract Template 


B-9 


61 % 



College Graduate 


Figure 9: Benefits of Educational Attainment to Society over Study Students’ Lifetime (Split into Costs and Net Benefits) 



Treatment: Costs ■ Treatment: Net Benefits Control: Costs ■ Control: Net Benefits 


Control mean is the unadjusted group average and treatment mean is the control mean plus the impact estimate. Statistically 
significant treatment-control differences at the p<0.05 level are shown by *. 
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Table 1: Background Sample Characteristics 



Whole 

Sample 

(N=716) 

Treatment 

Group 

(N=415) 

Control 

Group 

(N=301) 

T-C Difference 


Mean 

Mean 

Mean 

Difference 

P-Value 

Race & Ethnicity 

American Indian 

0.5% 

0.9% 

0.0% 

0.9% 

0.088 

Asian 

1.3% 

1.5% 

1.0% 

0.5% 

0.595 

Black 

21.3% 

21.6% 

20.8% 

0.8% 

0.892 

Hispanic 

5.6% 

5.9% 

5.3% 

0.5% 

0.788 

Multi-racial 

2.9% 

2.2% 

3.8% 

-1.6% 

0.330 

White 

68.4% 

68.0% 

69.1% 

-1.1% 

0.764 

Gender 

Male 

38.3% 

36.1% 

41.4% 

-5.3% 

0.797 

Age 

15.34 

15.32 

15.37 

-0.05 

0.128 

Exceptionality 

Disabled/ 

Impaired 

3.9% 

3.7% 

4.3% 

-0.6% 

0.647 

Gifted 

12.8% 

11.5% 

14.9% 

-3.3% 

0.381 

First Generation 
College 

45.9% 

44.0% 

48.5% 

-4.5% 

0.243 

Free/Reduced 
Price Lunch 

50.5% 

48.9% 

52.6% 

-3.6% 

0.372 

Retained 

3.5% 

2.2% 

5.2% 

-3.1%* 

0.024 

8“* Grade Achievement 

Math - z- score 

0.01 

0.02 

0.00 

0.02 

0.657 

Reading - z-score 

0.00 

0.04 

-0.04 

0.08 

0.278 

Math - pass 

82.2% 

85.2% 

78.1% 

7.12%* 

0.017 

Reading - pass 

97.8% 

98.8% 

96.6% 

2.19% 

0.048 

Algebra 1 - take 
up 

25.8% 

28.1% 

22.7% 

5.38% 

0.077 

Algebra 1 - pass 

97.2% 

96.6% 

98.2% 

-1.63% 

0.432 

Algebra 1 - z- 
score 

0.01 

-0.01 

0.04 

-0.04 

0.883 


Notes: 

“ The proportions are weighted by students’ probability of being selected into the ECHS. 

* This is the core analytic sample used for many outcomes and excludes students who could not be found in the 9* 
grade administrative data. 

Statistically significant treatment -control differences at the p<0.5 level are shown by *. 
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